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Abstract 

For a wide class of continuous-time Markov processes, including all irreducible hypoel- 
liptic diffusions evolving on an open, connected subset of R'', the following are shown to 
be equivalent: 

(i) The process satisfies (a slightly weaker version of) the classical Donsker-Varadhan 
conditions; 

(ii) The transition semigroup of the process can be approximated by a finite-state hidden 
Markov model, in a strong sense in terms of an associated operator norm; 

(iii) The resolvent kernel of the process is 'w-separable', that is, it can be approximated 
arbitrarily well in operator norm by finite-rank kernels. 

Under any (hence all) of the above conditions, the Markov process is shown to have a 
purely discrete spectrum on a naturally associated weighted Lao space. 
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1 Introduction and Main Results 

Consider a continuous-time Markov process $ = {^{t) t > 0} taking values in an open, 
connected subset X of M*^, equipped with its associated Borel a-field B. We begin by assuming 
that $ is a diffusion, that is, that it is the solution of the stochastic differential equation 
(SDE), 

d^{t) =u{<^{t))dt + M{<^{t))dB{t), t>0, $(0) = x, (1) 

where u = (ui, M2, . . . , n^)"^ : X ^ M'' and M : M'^ ^ M'' x M'^ are locally Lipschitz, and 
B = {B(t) : t > 0} is A;-dimensional standard Brownian motion. [Possible extensions to 
more general Markov processes are briefly discussed in Section 1.3.] Unless explicitly stated 
otherwise, throughout the paper we assume that: 



The strong Markov process $ is the unique strong solution of (1), with 
continuous sample paths. 



(Al) 



The distribution of the process $ is described by the initial condition <I>(0) = x € X and the 
semigroup of transition kernels {P* '■ t > 0}, where, 

P\x, A) := Px{^{t) e A} := Pr{$(t) G A \ $(0) = x}, x£X, AeB. 

Recall that P*, or any kernel P on X, acts as a linear operator on functions F : X — > M on the 
right and on signed measures v on (X, B) on the left, respectively, as, 

PF{x) = j F{y)P{x,dy), uP{A) = j u{dx)P{x, A), xeX,AeB, 

whenever the above integrals exist. Also, for any signed measure v on (X, B) and any function 
F : X — > M we write i^{F) := J Fdv, whenever the integral exists. 

The ergodic theory of continuous-time Markov processes is often most easily addressed 
by translating results from the discrete-time domain. This is achieved, e.g., in [6, 22, 23, 21] 
through consideration of the Markov chain whose transition kernel is defined by one of the 
resolvent kernels of defined as. 



oo 

at Tjt 



R^ = / e-^'PUt, a>0. (2) 
Jo 

In the case a = 1 we simply write R := Ri = e~^P^ dt, and call R "the" resolvent kernel 
of the process 

One of the central assumptions we make throughout the paper is the following regularity 
condition on the resolvent: 



The process $ has a continuous resolvent density, that is, there exists a 
continuous fuction r : X x X — > M, such that, 

R{x,dy) =r{x,y)dy, x,y eX. 



(A2) 
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For example, (A2) holds if $ is a hypoelliptic diffusion [17, 18]; see Theorem 2.1 below. The 
density assumption (A2) in particular implies that there exist continuous functions s,n: X — > 
IR+ that are not identically zero, and satisfy, 

R{x,dy) > s{x)n{y) dy, x,y€X. 

The function s and the positive measure defined by v{dy) = n{y)dy are called small, and this 
inequality is written in terms of an outer product as, R > s ® v] cf. [24, 20]. Without loss 
of generality (through normalization) we always assume that i^(X) = 1, so that v defines a 
probability measure on (X, S). 

Some of the results on ergodic theory require the following 'reachability' condition for 
it is a mild irreducibility assumption: 



There is a state xq G X such that, for any x G X and any open set O 
containing xq, we have, 

O) > for ah t > sufficiently large. 



(A3) 



Under (A3) we are assured of a single communicating class, since then the process is tp- 
irreducible and aperiodic with V'( ' ) •= R{xo, •): For all x G X and all A £ B such that 
R{xq,A) > 0, we have, 

P^{x,A) > 0, for all t sufficiently large. 

See [22, Theorem 3.3] and Proposition 2.2 below. 

Recall that the generator of $ is expressed, for bounded functions /i: X ^ M, as, 

d 

Vh[x) = Ui{x)——h{x) + iy^Si,(x)-- — —h{x), x eX, 
^ dxi ^ ■'^ dxi ax-\ 

or, in more compact notation, 

p = n- V + itrace(SV2), 

where S = MM'''. Rather than restricting attention to functions, we say that h: X ^ M is 
in the domain of V if there exists a function / : X — > M satisfying, 

P^h = h+ [ P'fds, t>0, 
Jo 

and we then write / = Vh. This definition is slightly stronger than the usual definition of the 
extended generator [7]. 

In this paper we consider approximations of the resolvent R and the semigroup {P*} using 
finite-rank operators. All of these approximations can be expressed in terms of a generator 
that is a finite-rank perturbation of the identity, of the form, 

N 



Approximating a Diffusion by a Finite-State HMM 



Draft copy June 1, 2009 



3 



where {Cj : 1 < i < N} is a finite collection of disjoint, precompact sets, Co is the complement 
X \ Ui<j<7vC'j, and {f,} are probability measures on {X,B) with each z/j supported on Cj. The 
constants k and {rij} are nonnegative, and the {vij} define a transition matrix on the finite 
set {1, 2, . . . , N}. Approximations of this form were first introduced in [16] to establish large 
deviation asymptotics and strong ergodic theorems for Markov chains and processes. The 
approximating semigroup is expressed as the exponential family, 

g* = e*^, t > 0, (4) 

where the exponential is defined via the usual power-series expansion. The family of resolvent 
kernels of the semigroup {Q*} is denoted T^, q > 0, where, 

/•oo 

= / e""*Q* dt. (5) 
Jo 

The main assumptions for the present results (assumptions (Al) - (A3) above and (A4) 
below) are a slightly weaker version of the classical Donsker-Varadhan conditions [3, 4, 5], 
and are naturally expressed in terms of the nonlinear generator; see also [15, 16]. Recall that 
Fleming's nonlinear generator [10] for the continuous-time Markov process $ is defined via, 

n{F) := e-^Ve^ , (6) 

for any function F : X — > M for which e^ is in the domain of D. Theory surrounding mul- 
tiplicative ergodic theory and large deviations based on the nonlinear generator is described, 
e.g., in [8, 31, 9, 16]. We say that the Lyapunov drift criterion (DV3) holds with respect to the 
Lyapunov function V : X —> (0,oo], if there exist a function W: X — > [1,cxd), a compact set 
C C X, and constants 6 > 0, b < oo, such that, 

n{V) < -6W + bic . (DV3) 

In most of the subsequent results, the following strengthened version of (DV3) is assumed: 

Condition (DV3) holds with respect to continuous functions V, W that 
have compact sub level sets. 

Recall that the sublevel sets of a function F : X ^ ]R_|_ are defined by, 

Cpir) = {x eX: F{x) < r}, r > 0. (7) 

Note that the local Lipschitz assumption in (1) together with (DV3) imply (Al), namely, 
that (1) has a unique strong solution $ with continuous sample paths; see [23, Theorem 2.1] 
and [26, Theorem 11.2]. 

Conditions (A1-A4) are essentially equivalent to (but weaker than) the conditions imposed 
by Donsker and Varadhan in their pioneering work [3, 4, 5]. Condition (DV3) is a generalization 
of the drift condition of Donsker and Varadhan. Variants of this drift condition are used in 
[1, 31, 25, 15, 12], and (DV3) is the central assumption in [16]. 

One important application of (DV3) here and in [16] is in the truncation of the state space 
- this is how we obtain a hidden Markov model approximation, where the approximating 



(A4) 
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process eventually evolves on a compact set. Wu, beginning with his 1995 work [30], has 
developed this truncation technique for establishing large deviation limit theory, as well as the 
existence of a spectral gap in the Lp norm, in a spirit similar to this paper and [16]. For recent 
bibliographies on these methods and other applications see [11, 12]. A significant contribution 
of the present paper and [16, 15], in contrast to the earlier work mentioned, is the introduction 
of the weighted Lex:, norm for applications to large deviations theory and spectral theory. In 
particular, for non-reversible Markov processes, the theory is greatly simplified and extended 
by posing spectral theory within the weighted L^o framework. 

The weighted norm is based on the Lyapunov function V from (DV3). We let v = e^ and 
define, for any measurable function g: X — > M, 

II II r \9{x)\ ^ y\ 

\\g\\v :=supi — — :xeX\; 

I v[x) i 

cf. [29, 14, 13] and the discussion in [20]. The corresponding Banach space is denoted := 
{g: X — >■ M : \\g\\v < oo}, and the induced operator norm on linear operators K : is, 

|||i^|||,:=sup{^M^:/.GL^^, ||% / o}. 

For example, the operator on induced by the resolvent kernel R will be shown to satisfy 
|||i?|||^ < oo under (DV3). An analogous weighted norm is defined for signed measures v on 
(X, B) via, 

|li.|l„:=sup{i|^:/i€L^, \\h\U^o]. 

All of the approximations in this paper are obtained with respect to ||| • 1,^. Our main results 
are all based on Theorem 1.4 below, which establishes conditions ensuring that the semigroup 
{P*} of the process $ can be approximated (in this weighted operator norm) by a semigroup 
written in terms of finite-rank kernels. In particular, our main result, given below, states that 
the Donsker-Varadhan condition (DV3) holds if and only if the process $ can be approximated 
by an HMM in operator norm. 

Theorem 1.1. [(DV3) HMM approximation] For a Markov process ^ onX satisfying 
conditions (Al), (A2) and (A3), the following are equivalent: 

(i) Donsker-Varadhan Assumptions: Condition (DV3) holds in the form given 
in (A4). 

(ii) HMM approximation: For each e > and 6 e (0, 1), there exists a continuous 
function v with compact sublevel sets (possibly different from the function v in (i) ), 
and a semigroup {Q*} of the form (4) with generator £ given in (3) and with an 
associated family of resolvent kernels {Tq} as in (5), satisfying the following: 

(a) Resolvent approximation: The resolvent kernels (2) and (5) satisfy, 



Ra-Ta\l<e, 6<a<6-\ 
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(b) Semigroup approximation: For each g G LJ^ satisfying V'^g G L"^, 

WP'g - Q'g\\v < e{\\g\U + WV^gWv), t > o. 

(c) Invariant measure approximation: The Markov processes $ and ^ have 
unique invariant probability measures vr and w , respectively, satisfying, 

llvr — w\\y < e. 

The proof of the theorem is contained in Section 3: The imphcation (i) ^ (iia) is con- 
tained in Proposition 3.3; the imphcation (i) ^ (iib) fohows from Proposition 3.5 combined 
with Proposition 3.6; and the imphcation (i) =^ (iic) is given in Corohary 3.7. Finahy, the 
imphcation (ii) ^ (i) follows from Proposition 2.4. 

Before formalizing these technical results we consider the structure of a Markov process 
with generator of the form given in (3), and show that it indeed admits a representation as a 
finite-state hidden Markov model. 

1.1 Hidden Markov model approximations 

Recall that the generator D of $ will be approximated by a generator £ of the form given in 

(3). Let Y denote the compact set Y := IJ^j^ Cj. Then the continuous-time Markov process, 
say ^ = {^{t) : t > 0}, with generator £ and with corresponding transition semigroup {Q*} 
defined in (4), has state space X, and can be realized either as a pure jump process, or as a 
hidden Markov model, or HMM. 

For a given initial condition ^'(0) = x G Cj for some i 7^ 0, consider independent 
exponential random variables with respective means equal to {(Krij)"^ : I < j < N}. The 
first jump after time tq := is given by the minimum of these exponential random variables, 
denoted ri. Similarly, given, ^(0) = x G Co, the first jump time ri after time tq := is given 
by the value of an exponential random variable with mean Letting j denote the index 
corresponding to the minimizing exponential random variable (or taking j = 1 if x G Co), we 
have ^'(t) = X for = To < t < Ti, and ^'(ti) ~ Uj. This procedure is continued to define a 
sequence of sampling times {tj} along with the jump process 

To see that ^ can be viewed as an HMM we first present a simplified expression for the 
semigroup {Q*}. The proof of Proposition 1.2 is given in Section 3.3. 

Proposition 1.2. Consider the process with generator £ as in (3) and semigroup {Q*} 
in (4). If the initial state ^'(0) is distributed according to some probability measure ^(0) ~ fj, 
of the form fi = YliLiPi^i' where the vector p = {pi,p2, ■ ■ ■ ,Pn) G ^+ satisfies YlPi = 1j then 
the distribution fiQ*- of^(t) at time t > can be expressed as, 

liQ^ = ^Pi{t)vi, t>0, where p{t) = e~'^^^~''^^p , 

and K,r = {rij} are the coefficients of the generator £ in (3). 

For the HMM construction, let / denote a finite-state, continuous-time Markov chain, with 
values in {0, 1,2,..., A^} and rate matrix qij := nrij for i 7^ j, and qu := — Ylj^i liji where we 
define roi = 1 and roi = for all i 7^ 1. Written as an [N + 1) x (A^ -|- 1) matrix, this becomes 
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q = —k{I — r). The process / is the hidden state process, and, conditional on /, we define the 
observed HMM process, denoted T = {T(t)}, as follows. Letting {rj} denote the successive 
jump times of /, T(t) is constant on the interval t € [rj, Tj+i), and satisfies for each A & B and 
f = 0,1,2,..., 

P{T{Ti) G A I T{t),t < n; Iit),t < Ti- lin) = k} = P{T(ri) G A \ = k} = v^iA). 
An immediate consequence of the definitions is that ^ can be expressed as an HMM: 

Proposition 1.3. Suppose that ^'(0) ^ Ui for some i > 1, and that the HMM is initialized in 
state i, i.e., /(O) = i. Then the jump process ^ and the HMM T are identical in law. More 
generally, if ^'(0) = x £ Ci and /(O) = i, for some i = 0,1, . . . , N , then the jump process ^ 
and the HMM T are identical in law following the first jump, 

{^(t):t>ri} =*{T(t):t>ri}. 
1.2 Separability and the spectrum 

The key property we will use to establish that a process $ can be approximation by an HMM 
as in Theorem 1.1 will be the "f-seperability" of its resolvent R. Following [16] we say that 
a kernel K is v-separable with respect to some function v : X ^ [l,oo), if |||i^|||^, < oo and, 
for each e > 0, there exists a compact set Y C X and a finite-rank, probabilistic kernel T 
supported on Y, such that fK — Tf^ < e. By 'finite-rank' we mean there are functions {si}, 
measures {I'j}, and nonnegative constants {6ij} such that, 

N 

A kernel T is 'probabilistic' if T{x, X) = 1 for all x G X. 

Our next result gives an alternative characterization of the Donsker-Varadhan condition 
(DV3), showing that it is equivalent to ?;-seperability of the resolvent. A similar result in 
discrete time appears in [16]. The implication (ii) =^ (i) is contained in Proposition 2.4. The 
forward implication (i) =^ (ii) follows from Proposition 3.1. 

Theorem 1.4. [(DV3) <;=^ Separability] For a Markov process ^ onX satisfying condi- 
tions (Al) and (A2), the following are equivalent: 

(i) Donsker-Varadhan Assumptions: Condition (DV3) holds in the form given 
in (A4). 

(ii) u-Separability: The resolvent kernel R is v-separable, for a continuous function 
v with compact sublevel sets, possibly different from the one in (i). 

The spectrum S{K) C C of a linear operator K on LJ^ is the set of z G C such that 
the inverse [Iz — K]~^ does not exist as a bounded linear operator on L^. As an immediate 
corollary to Theorem 1.4 and Proposition 3.3 we have: 

Theorem 1.5. [(DV3) =^ Discrete Spectrum] Let ^ be a Markov process satisfying con- 
ditions (Al) and (A2). If ^ also satisfies the drift condition (DV3) in the form given in (A4), 
then the spectrum of the resolvent kernel is discrete in . 
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1.3 Extensions 

Further connections between (DV3), f-seperability, multiplicative mean ergodic theorems, and 
large deviations for continuous-time Markov processes will be considered in subsequent work, 
generalizing and extending the discrete-time results of [16]. In particular, under (DV3), the 
process $ is "multiplicatively regular" and satisfies strong versions of the "multiplicative mean 
ergodic theorem." These results, in turn, can be used to deduce a large deviations principle 
for the empirical measures induced by Moreover, the rate function can be expressed in 
terms of the entropy, as in [3, 2, 16]. 

The technical arguments used in the proofs of all the central results here can easily be ex- 
tended beyond the class of continuous-sample-path diffusions in W^. Although such extensions 
will not be pursued further in this paper, we note that the assumption (Al) can be replaced 
by the condition that $ is a nonexplosive Borel right process (so that it satisfies the strong 
Markov property and has right-continuous sample paths) on a Polish space X. Assumptions 
(A2) and (A3) can be maintained as stated; the conclusions of Proposition 2.2 continue to 
hold in this more general setting. Assumption (A4) can also be maintained without modifica- 
tion, where V denotes the extended generator for the process. The resolvent equations (12) 
hold in this general setting, which is what is required in the converse theory that provides the 
implication (ii) =^ (i) in Theorem 1.1. 




The remainder of the paper is organized as follows. The following section develops results 
establishing approximations between the process $ and a simple jump process. This is a 
foundation for Section 3 that establishes similar approximations with an HMM. 
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2 Resolvents and Jump-Process Approximations 

We begin in this section with an approximation of the process $ by a pure jump-process 
denoted evolving on the state space X. The fixed constant k > denotes the jump rate. 
The jump times {ri : i > 0} define an undelayed Poisson process: tq = 0, and the increments 
are i.i.d. with exponential distribution and mean k~^. At the time of the ith jump we have 
$'^(Tj) ~ kRi^{x, ■ ), given that <I>'^(Tj_i) = x. This process is Markov, with generator, 

V^:=k[-I + kR^]. (9) 

Throughout this section it is assumed that < oo, with v being continuous, with compact 

sublevel sets. Hence the generator also has finite norm. Before considering the relationship 
between and we first review some background on Markov processes, diffusions, and 
their resolvents. 

2.1 Densities, irreducibility and separability 

We begin with some well-known properties of the resolvent kernel. Theorem 2.1 illustrates 
that the density assumption (A2) is often easy to verify. Its proof follows from [17, 18], and 
related results are obtained in [19, 27, 28]. 

Theorem 2.1. Suppose that the Markov process $ satisfies (Al), with u and M being smooth, 
u,M C°° . If in addition the generator D of ^ is hypoelliptic, then $ has a continuous 
resolvent density, i.e., (A2) holds, and in fact the density r is smooth, r G C°°. 

The density condition (A2) combined with the existence of a Lyapunov function as in 
(DV3) implies ergodicity. Recall that a Markov process $ with a unique invariant probability 
measure vr is called f-uniformly ergodic for some function u : X ^ M, if there are constants 
6o > 0, i?o < such that, 

IIP* - 1 7r|||^ < e-^o-^'o*, t > 0. 

Proposition 2.2. // conditions (Al), (A2) and (A3) hold, then the Markov process $ is ip- 
irreducible and aperiodic with ■ ) ■.= R{xq, ■ ), and all compact sets are small. If, in addition, 
(DV3) holds, then the process is v-uniformly ergodic with v = e^ . 

Proof. Under (Al) and (A2) the Markov process is a T-process [22]. This combined with (A3) 
easily implies i/^-irreducibility with ^(•) = R{xq, •). Under (A3), for any set A satisfying 
'tp{A) > 0, we have P''{x,A) > for all t > sufficiently large. The proof is similar to the 
proof of Proposition 6.1 of [22]. Hence the process is aperiodic. To see that all compact sets 
are small, we note that all compact sets are petite by [22, Theorem 4.1]. Under aperiodicity, 
petite sets are small; this is proved as in the discrete-time case [20, Theorem 5.5.7]. 

To see that $ is 7;-uniformly ergodic note that, under (DV3), we have, for a possibly larger 
constant b', 

Vv < -6v + b'ic. 

This is condition (V4) of [6], and hence the conclusion follows from the main result of [6]. □ 
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2.2 Resolvent equations 

Recall the construction of the process with generator as in (9). We denote the semigroup 
of by P* := e^^", t >0, and its associated family of resolvent kernels by Rn^a'- 

POO 

R^,a-= e-'^^Pidt, a>0. (10) 
Jo 

Proposition 2.3 states the resolvent equations, and establishes some simple corollaries. Parts (i) 
and (iii) follow directly from (ii), and (iv) follows from (i). Part (ii) is stated, e.g., in [7]. 

Proposition 2.3. Suppose the process $ satisfies (Al) and is the jump process with 
generator as in (9) . Then, for any positive constants a, /9 we have: 

(i) The resolvent equation holds, 

Ra = Rp + iP- a)RpRa = Rp + {P- a)R^Rp. (11) 

(ii) For each a > and any measurable function /i: X ^ M for which R\h\ is finite- 
valued, the function R^h is in the domain ofD, and, 

VRah = aRah - h. (12) 

(iii) The resolvent of satisfies the analogous identity, 

T^kRk., ah = aRf,^ ah - h, (13) 
whenever each side of this equation is well-defined. 

(iv) The generators for $ and are related by, 

V, = V[kR^] = [kR^]V. (14) 

As a first application of the resolvent equations, the next result shows that u-separability 
implies the drift condition (DV3). Its proof is in the Appendix. 

Proposition 2.4. Suppose that the Markov process $ satisfies conditions (Al) and (A2), and 
that its resolvent kernel R is v-separable for some continuous function v.X^ [l,oo) with 
compact sublevel sets. Then (A4) holds for some V^, W- on X, and the function VL is in L^. 

Before exploring the resolvent equations further, we introduce a new drift condition as a 
relaxation of (DV3). The proof of Lemma 2.5 is trivial from the definition of (DV3). 

Lemma 2.5. Let ^ be a Markov process satisfying (Al). // (DV3) holds, then the follow- 
ing drift condition also holds: There is a function f : X — > [l,oo) and a positive constant b' 
satisfying, 

Vv<-v + b'. (15) 
Specifically, under (DV3) the function v can be taken to be eY , and, 

b' := by := b sup v{x). 
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Next we apply the resolvent equation (12) to obtain an explicit bound on the u-norm of 
the resolvent based on the drift condition of Lemma 2.5: It implies that is finite, and 

we obtain the following explicit bounds: 

Lemma 2.6. Suppose the process $ satisfies (Al) and the drift condition (15) of Lemma 2.5. 
The following hounds then hold: 

|||(a^a)"|IL < 1 + b', for alln>l, a> 0; 

■k{v) < b' , for any invariant probability measure vr. 

Proof. Under (15) we have, 

VRaV < RaVv < Ra[-v + b'] = -R^v + b'a~^. 

We also have from (12) the identity VRa = aRa — I, giving aRaV — v < —RaV + b'a~^, so 
that, 

aRaV <{l + a~^)-^{v + b'a~^). (16) 
Iterating this bound we obtain, for any n > 1, 

n 

{aRaTv < (1 + a-^yv + b'a'^ ^,{1 + a-^)-'' 

k=l 

<v + b'a-^[l - (1 + a-^)]-^ =v + b'. 

Hence {aRa)^v < (1 + b')v, which is the first bound. 

The second follows from (16) and the Comparison Theorem of [20], which gives, 

Tr{v) < {l + a-^)~\7T{v) +b'a-^). 

Rearranging terms gives (1 + a^^)7r{v) < tt{v) + b'a~^, or 7:{v) < b' as claimed. □ 

The resolvent equation implies that [la — V] is a left inverse of Ra for any a > 0, in the 
sense that, [la — VjRaf = f for an appropriate class of functions /. While Ra cannot be 
expressed as a true operator inverse on the space L^, it is in fact possible to obtain such a 
representation for i?K,ci- This is made precise in the following lemma, which is used in the 
proof of the main result of this section, Proposition 2.9. 

Lemma 2.7. Suppose the process $ satisfies (Al) and the drift condition (15) of Lemma 2.5. 
Then, for any a > 0, 

oo 

= [al - V^r' = [al - k{kR^ - = ^ (1 + aK~Y^{^R^r+\ (17) 

n=—l 

where the sum converges in L^. Moreover, 

\lR.,a\l<a-\l + b'). (18) 
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Proof. For any n > 0, k > 0, we have the bound IIK'^-Rk)"'''^!!^, ^ (1 + b'), from Lemma 2.6. 
The representation (13) imphes that the inverse can be expressed as the power series (17), 
which is convergent in L^. Since al — T)^ is a left inverse of RK,a, it then follows that 

To estabhsh the bound (18) we apply the triangle inequality, 

oo 

^ ' 71= — 1 

Using once more the bound < (1 + b'), and simplifying the expression for the sum 

in the following bound, 

^ (1 +^')(^f^ (i 1^1+1^-1)^0 ' 

we obtain the bound in (18), as claimed. □ 



The range space of a kernel K on X x is denoted TZy{K) = {Kg : g £ L^}. The following 
is a mild refinement of Proposition 2.3. 

Lemma 2.8. Suppose that the process $ satisfies (Al) and that its resolvent kernel R: LJ^ — 
for some v : X — > [l,oo). The following are then equivalent for a measurable function 
g:X^R: 

(i) g e n„{R); 

(ii) g € TZy{Ra) for each a > 0; 

(iii) geLl, andVgeLl,. 

Proof. The equivalence of (i) and (ii) follows from (11). If (i) holds then g = Rh for some 
h £ and hence, 

Vg = Rh-h£ L^, 
which establishes the implication (i) =^ (iii). Conversely, under (iii) we have, 

RVg = Rg - g, 

SO that g = Rh with h : = g — Dg € L^. Hence (iii) =^ (i). □ 



2.3 Resolvent approximations 

Under (DV3) or, more generally, under the weaker drift condition (15) introduced in Lemma 2.5, 
we obtain the following strong approximation for the resolvent kernels: 

Proposition 2.9. Suppose the process $ satisfies (Al) and is the jump process with 
generator defined in (9). // $ satisfies the drift condition (15) of Lemma 2.5, then, for 
each a < k: 
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Proof. We first obtain a power series representation for Ra not in terms of its generator, but in 
terms of the resolvent kernel i?^- The resolvent equation (11) with f3 = k and a > arbitrary 
gives [I — {k — a)RK]Ra = Rk- Since < a < k and Rk{x, X) = for each x, it follows that 
Ra can be expressed as the power series, 



R, 



« = [/-(!- aK~^)KR^]~^R^ = - ^(1 - aK-^y'{KR^) 



K 
n=0 



Lemma 2.6 gives the uniform bound, |||(K-Rft)"''^^|||t, < (1 + b'), which implies that this sum 
converges in L^. 

Applying Lemma 2.7, we conclude that the difference of the two resolvent kernels 
and Ra can be decomposed into three terms: 

^ oo 

R.,a-Ra = ( , --) V(l + aK-^)-"W"+^ (19a) 

^ ' n=0 

^ oo 

+ - + o'^"')"" - (1 - «^"')") {^^R.T^^ (19b) 



.(K; + a)2 

To bound the first term (19a) we apply Lemma 2.6: 



n=0 

+ + )!■ (19c) 

' " ' n= — 1/ 



^ oo 

IIRHS of (19a)|||, < (1 + b') - -| E(l + 



-1 \— n 



n=0 



(1 + b')(- - , f 1 - (1 + a/^"') 



(1+6')'" + ° 



< 



{k + a)2 
2(1 + b') 



-l\n 



~ K + a 

The next inequality also uses the bound |||(«:i?K)"|||t, < 1 + b': 

1 °° 

IfRHS of (19b)|||,, < (1 + fc-i)- V((l + aK-^y - (1 - an 

n=0 

= (1 + b')-([l - (1 + a^-i)-!]-! - [1 - (1 - aK-i)]-i 

K V 

_ 1 + 6^ 

The final term (19c) is elementary: 

1 



IfRHS of (19c)|||„ = -^(1 + aK-^] 



{k + a)2 K + a 

Substituting these three bounds completes the proof. □ 
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3 Separability 

In this section we develop consequences of the separability assumption. In particular, we 
describe the construction of an approximating semigroup {Q*} with generator of the form 
given in (3), as described in Theorem 1.1. This is accomplished in four steps: 

(i) First we note that under (DV3) the resolvent kernel of $ can be truncated to a 
compact set; see Lemma A. 3. 

(ii) Then we argue that, again on a compact set, R can be approximated by a finite-rank 
kernel T; see Proposition A. 4. 

(iii) We next prove that the generator, ■= k[—I + Ki?^], of the jump process con- 
structed in Section 2, can be approximated by a generator £ of the form (3), 



as long as K > is chosen sufficiently large. This key result is described in Proposi- 
tion 3.1. 

(iv) Finally we show that the transition semigroup {-P*} of the original process $ can 
be approximated by the semigroup {P^} of the jump process (Proposition 3.5), 
and that the semigroup {P^} can in turn be approximated by the semigroup {Q*} 
corresponding to an HMM with a generator £ as above (Proposition 3.6). 

3.1 Truncations and finite approximations 

Let $ be a Markov process satisfying condition (Al), with generator T> and associated resolvent 
kernels {Ra}- Recall the definition of the corresponding jump process in the beginning of 
Section 2, with generator and associated resolvents 

Our result here shows that condition (DV3) implies that the generator of the jump 
process can be approximated by a generator £ as in (3). The proof is given in the Appendix. 

Proposition 3.1. Suppose the Markov process $ satisfies conditions (Al), (A2). // (DV3) 
holds as in assumption (A4), then, for each n > and any e > 0, there exists a generator £ 
of the form given in (3), such that all the rij are strictly positive, and the generator of the 
jump process can be approximated in operator norm as, 



From Proposition 3.1 we have a generator £ of the form (3), and with Q* := e , t > 0, being 
the associated transition semigroup, the corresponding resolvent kernels {T^} are defined, as 
usual, in (5). Using the approximation of the generator £ in (20), we next show that the 
kernels {Ta} can be expressed as operator inverses, in a way analogous to the representations 
obtained in Lemma 2.7 for the resolvents 



N 




\V.-£\l<e, 



(20) 



with V := e 
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Lemma 3.2. Suppose that the assumptions of Proposition 3.1 hold, and choose k > and 
eo > such that — f|||^ < eq. Then the resolvent obtained from the semigroup {Q*} can he 
expressed as an inverse operator on L"^: For all a > (1 + 6,j)eo, 

T^ = [Ia-£r\ 

where by is as in Lemma 2.5. Moreover, for all such a we have the norm bound, 

HIT III < ^ f^l^ 

Proof. Note that we already have from the resolvent equation the formula [la — £]Ta = I on 
L^. It remains to show that [la — £] admits an inverse. We can write, on some domain, 

[Ia-£]~^ = [Ia-V^ + V^-£]-^ = R,,^[I + {V^-£)R^^^Y\ 

The right-hand-side admits a power series representation whenever — £)RK,a III, < 1. In 
fact, under the assumptions of the Lemma, using the bound in Lemma 2.7 we have, 

- £)R^,a\l < WD^ - £\l ■ \lRK,a\l < eo(l + K)/a < 1, 
and the resulting bound is precisely (21). □ 



Our next result shows that w-separability implies that each of the resolvent kernels Ra can 
be approximated by the kernels {Tq,} obtained from a 'finite-rank' semigroup. Specifically, Ra 
will be approximated by a resolvent Tq of the form (5), where the transition semigroup {Q*} 
is that of a Markov process with generator £ as in (3). 

Proposition 3.3. Under the assumptions of Proposition 3.1, for each e > and 6 G (0,1), 
there exists a generator £ of the form given in (3), such that the corresponding resolvent kernels 
{Ta} defined in (5) satisfy the following uniform bound: 

\lRa-Ta\l<e, 6<a<6^\ 

Proof. To establish the uniform bound in operator norm, first we approximate R^ by Rf.^^ a • 
Under (DV3), Lemma 2.5 implies that we can use Proposition 2.9 as follows: We fix /t > 
such that the right-hand-side of this bound is no greater than ^e, giving, 

\lRK,a-Ra\l<^e, a<6-^ (22) 

We now invoke Proposition 3.1: Fix an operator £ of the form (3) satisfying, 

- £\t < ^0, 

where Eq £ (Oi^) is to be determined. Lemma 2.7 and Lemma 3.2 give, 

Ta = [Ia-£]-\ R^^a = [Ia-V^]-\ 
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Hence the difference can be expressed, 

— [^Q Rk, «] \£ "^kI-^k, a ~\~ Rk, a[£ "^kIRk, ai 

and applying the triangle inequality together with the sub-multiplicativity of the operator 
norm, 

\Ta — RK,a\v — lll^a — -^re, allUi'? — ^^kIILiI-Rk, + lll-Rfc, olll^lll^ — 

Applying Lemma 2.6 gives the bound |||aiiK,a||li, < (1 + b'), and hence for a € [6, d~^], 

l\£-V^l,\lR^,a\l<eo{l + b')/6. 

Assuming that eo > is chosen so that the right-hand-side is less than one, we can substitute 
into the previous bound and rearrange terms to obtain, 

HIT - /? Ill < il^^.-i'li^-^^ll < u ,,2. (1 + ^0^ 

\Ua it:.,«|||,< <(eo/0 

Choosing £q = ^{1 + then gives, 

- RK,a\l < \e \ <\e, a G [5,6~\ 
This combined with (22) and the triangle inequality completes the proof. □ 



3.2 Ergodicity 

To establish solidarity over an infinite time horizon we impose the reachability condition (A3) 
throughout the remainder of this section. Recall the construction of the approximating HMM 
process ^ in Section 1.1, and the definition of w-uniform ergodicity from Section 2.1. 

Proposition 3.4. Suppose the process $ satisfies conditions (Al) - (A4), so that, in partic- 
ular, $ is v-uniformly ergodic with v = by Proposition 2.2. Then: 

(i) For each k> 1, the jump process is v-uniformly ergodic, with v = e^ . 

(ii) The HMM process ^ is v-uniformly ergodic, with v = e^ . 

Proof. To establish (i) we first demonstrate that is irreducible and aperiodic. If ■0 is a 
maximal irreducibility measure for then Lemma 2.7 implies that ip -< Rk{x, •) for each 
X. This implies that the chain with transition kernel k-R^ is ■0-irreducible and aperiodic. 
Irreducibility and aperiodicity for is then obvious since it is a jump process with Poisson 
jumps, and jump distribution kR^- 

To complete the proof of (i) we establish condition (V4) of [6]. Suppose that we can find 
(5k > 0, 6k < CO satisfying the bound. 



V^v < -S^,v + 6k. 



(23) 
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We would then obtain, 

V^v < -l6^v + bJc^, 

where = {x : ^Si^v{x) < b^}. The sublevel set is compact, and Proposition 2.2 states 
that compact sets are smah, so this imphes that the jump process is f-uniformly ergodic. 
Hence it remains to estabhsh (23). 
We begin with the bound, 

Vv < -5v + bic, 

which holds under (DV3) because W > 1 everywhere. Applying to both sides we obtain 
from the resolvent equation (12), 

kR^^v -V < -6R^v + bRtJc < -5Rt,v + bK~^ . 

On rearranging terms this gives, 

kR^v < (1 + 5k"^)~^{v + 6k"1), 

and thence, 

1 , 

kRkV — V < —- V + 7 5. 

+ K + K 

From the definition of the generator for the jump process we conclude that the desired bound 
holds, 

5k k 
Vf^v = k\kR^,v -v\< V + 6. 

+ K + K 

This is (23) with 6^ = 5k/ [6 + k) and 6^ = + ^) 

We now prove the analogous resuh for ^ to establish (ii): ^-irreducibility and aperiodicity 
are immediate by Proposition 1.3 and the fact that all rij in the definition of £ are strictly 
positive, from Proposition 3.1. To show that ^ is 7;-uniformly ergodic simply note that, by 
the definition of £, 

£v = —Kv + kTv < —KV + Kbly, 

where b := sup^gy ''^(^)- Again, this is a version of condition (V4) of [6], and the conclusion 
follows from [6]. □ 



3.3 Semigroup approximations 

To establish the semigroup approximations we begin with the representation result for the 
semigroup {Q*} contained in Proposition 1.2. 

Proof of Proposition 1.2. It suffices to prove the result with ^ = Vi for some i\ the general case 
follows by linearity. 

The power series representation of implies that ViQ^ can be expressed as a convex 
combination of {uj} for each t, 

N 

z^iQ* = ^£>ii(i)i^j, *>0. (24) 
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An expression for the coefficients {gij{t)} can be obtained from the differential equation, 



as follows: Writing v^S = k(— + Ylf=i '''ij^j)^ conclude that, for any t > 0, 



d_ 
It 



N N JV . 



k=l 3=1 

Therefore, the coefficients q G appearing in (24) satisfy, 



2 



d 



N 



-^Sik{t) = K -Qik{t) + ^rijgjk{t) 



Given the initial condition Qij{0) = I, the solution to this ODE is given by, Q{t) = e '^^^ 

t > 0, as required. □ 



Next we obtain an approximation bound between the semigroups corresponding to $ and 

Proposition 3.5. Suppose that $ satisfies conditions (Al) - (A4). Then there exists b, 
depending only on $ such that, 

WP'g - PigWv < b,K-'^\\v^g\U, t>o, k>i. 

Proof. Fix g £ L"^ satisfying Vg G LJ^, and denote the difference eg{t) = P^g — Pj^g. We have 
for any t, 

^^ey{t) = VP'g-V^Pig 

= V^[P'g-Pig] + [V-V,]P'g 
= V,[P'g-Pig]-K~\V,V]P'g, 
where the final equation follows from (14). This can be solved to give, 

eg{t) = £3(0) + f e'^-[-Hi-^V^VP'-'g] ds. 
Jo 

Substituting P^ = e^'^"-, £g(0) = 0) ^'^'^ writing h = Vj^Dg, we obtain 



Egit) = -K-^ [ P^P^-'hds 
Jo 



We have used here the fact that the operators P*, P*, and all commute. 

We have vr(/i) = 7r(P*~*/i) = 0, so that by Proposition 3.4 we have the bound, 

\\P^P^"'h\\„ < e^''~'"''\\P^~'h\U 
_ J2Bo-hot 



(25) 
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Substituting into (25) gives ||e3(t)||t, < K~^\\h\\^{te'^^°-'">^) . We have h = V^Vg, and hence 
the generator relationship (14) and the generator bound in Lemma 2.6 give, 

\\h\U < iKR^jV^gWv < (1 + bc~^)\\V^g\U. 
Finally, substituting this into the previous bound completes the proof. □ 

Similar arguments provide approximation bounds for the semigroups corresponding to 
and where the latter is denoted {Q*} and defined in (4). 

Proposition 3.6. Suppose that $ satisfies conditions (Al) - (A4). Then there exists b, 
depending only on $ such that, 

\\Pk9 - Q^gWv < b.e\\g\l, 

Proof. The proof is similar to the proof of Proposition 3.5: We fix G L^, and denote the 
error by 

eg{t) = Pig - Q'g, t > 0. 
Taking derivatives as before gives, 

d 

eft' 

which can be solved to give. 



;£g{t) =V^eg{t) + [V^-S]Q'g 



Egit) = SgiO) + fp^[V^ - £]Q'~'gds. 
Jo 



We have £§(0) = 0. Moreover, [P^ — f]l = 0, which implies that [V^ — £]g = [V^ — £][g — 
zu{g)]. Here, zu denotes the unique invariant measure of the process guaranteed to exist by 
Proposition 3.4. Hence, 

rt 



\^9 



= WT^n - £\l [ III^'kIIUKQ*"'^ - 1 ® ru)g\\, ds. 
Jo 

Proposition 3.4 implies the bound, 

< \IK - 1 ^ vrlll, + ill ® 7r\l < e^"-""* + 7r{v), 
so that the previous bound gives, 

\\eg{t)h < I'D. - SIMU ^*(e2^»-^°* + ^(z;)e^"-^°(*-^)) ds 

< - £\l\\g\U (te2^°-''°* + 7r(t;)6o ie^«^ 

Lemma 2.6 gives 7r(i;) < 6c~^, which completes the proof. □ 

The following bound is an immediate consequence of the last Proposition. 

Corollary 3.7. Under the assumptions of Proposition 3.6, for each e > we can construct 
the approximating process ^ described in Section 1.1 so that the the Markov processes $ and 
^ have unique invariant probability measures vr and w, respectively, satisfying, 

llvr — zuW^u < e. 
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A Appendix: Proof of Proposition 2.4 

Under the separability assumption we can find, for each n > 1, a compact set Y„ and a kernel 
Tn supported on Y„ satisfying fR — T„|||^ < 2~". Writing v„ = vlyc we have WvnWy = 1 and 
TnVn = 0. Consequently, for each n > 1, 

Rvn = {R- Tn)vn + T^v^ = {R - Tn)vn < jR " T„,|||^-i; < 2~"i;, n > 1. (26) 

The desired solution to (DV3) is constructed as follows. First define the finite-valued function 
on X, 

oo oo 

U-:=v + ^Vn= (l + ^Iyc^ 

n=l n=l 

Let V- = Ru-, and define V- = log(t'_). Applying (26), we conclude that V- E L^, with the 
explicit bound, 

oo 

\\v.\u<\lR\l + Y,\\Rynl<\lR\l + '^- 

n=l 

Moreover, the continuity of VL follows from (A2). 

It follows from Proposition 2.3 that the resolvent equation holds, Vv^ = — u_, and 
consequently, recalling the nonlinear generator (6), 

n{V-) = {v-y^Vv- = 1 - U-/V-. 

The function u-/v^ is continuous, with compact sublevel sets on X since f_ G L^. Writing 
W = m.ax{u- /v- — 1, 1) and C = {x : W{x) < 1} then gives, 

n{v.) < -w + 2ic, 

which is a version of (DV3). The function W is not continuous. However, it has compact 
sublevel sets, so there exists a continuous function : X ^ [1, oo) with compact sublevel 
sets, satisfying W- < W everywhere. The pair {V-, W-) is the desired solution to (DV3). □ 

Proof of Proposition 3.1 

Before giving the proof, we state and prove some preliminary results. The assumptions of 
Proposition 3.1 remain in effect throughout this subsection. 

The next lemma is an immediate consequence of Proposition 2.3. 

Lemma A.l. IfDh < —f + g for nonnegative functions (h, f,g) and g is bounded, then, 

R{f + h) <h + Rg. 

On setting h = v = e^ , f = 6Wh, and g = bich in Lemma A.l we obtain the following 
bound: 

Lemma A. 2. Under (DV3), with v = e^ , we have, 

RIwiV < V + bso, 

where Wi = 1 + 5W, sq = RIcv and for any function F , Ip denotes the multiplication kernel 
lF{x,dy) = F{x)5x{dy). 
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For each r > 1, we define the compact sets, 

Cr = Cy{r)nCw{r), 

in the notation of equation (7). From the assumption that V and W are continuous with 
compact sublevel sets, we obtain, 

hm inf V{x) = hm inf W{x) = oo. (27) 



r 



The above bounds on the resolvent will allow us to approximate i? by a kernel supported on C, 
for suitably large r > 1. To that end, we choose and fix a continuous function Wq : X — > [1, o6) 
in L^, satisfying HVFo = 1, and whose growth at infinity is strictly slower than W"^ in the 
sense that, 

hm ||Ty2lcM.(r-)=lliy = 0. (28) 



r— >oo 

This can be equivalently expressed 



lim sup 



^o(^) , 



0. 



The weighting function is simultaneously increased to 

tiQ = Wqv. 

The following Lemma justifies truncating i? to a compact set. 

Lemma A. 3. Under (DV3) he resolvent kernel R satisfies < oo? fl^c^? 

lim WlwoiR - ICrRICr)Iwo\l, = 0. (29) 

Proof. Lemma A. 2 implies that is finite as claimed: We have the explicit bound 

WRIwlL < 5-1(1 + 6||soD- The limit (29) is also based on the same lemma. Starting with the 
identity R - Ic^RICr = ICrR^C^ + la^R we obtain, 

IWo{R - ICrRICr)IWoVQ = IWo{R " /c,- i^-^C,- ) ^2 ^ 

= Iwo Vcr RIcf Iw^ ] ^ + ^Wq [Icf. RIwo ] ■ 

These two terms can be bounded separately. For the first term on the right-hand-side consider 
the following, 

[Ic^RIc^Iws]v < WlcrRIc^IwlsrV < WRIwlsrV, 

where £r '■= ^'^Pxgc^ Wo{x)W~^{x). Multiplying both sides by Wq then gives, 

[IwoICrRIciiIwo]vo < \lRIw\l^£rVo, r > 1, 

which means that WlwoICrRIc^IWoWvo - llR^wWy^r for each r. 

Bounds on the second term in (30) are obtained similarly through a second truncation. 
Write, for any n > 1, 

[Ic^RIw§]v = [Ic-RIwg^cJv + [Ic^RIw2lc^]v. 
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Arguing as above we have |||/c'ci2/^y2lcc < |||i?/iy|||t,en- Moreover, W^v < Wv < v? on C^, 
which gives, 

[Ic^m^f^v < n^Icc + WRIwtenV. 
Multiplying both sides of this equation by Wq gives, 

SO that 

|||%o^C.^^%olll.o < ^AUc^Wol^ + WRIwlSn- (31) 



And also, 



IN. N, ^0(2;) 1 1 

|||-fc^;W^o|||„o = sup —j-^ = sup < -. 



This combined with (27) implies that (31) can be made arbitrarily small by choosing large n 
and then large r. □ 



Proposition 3.1 will be seen as a corollary to the following more general bound: 

Proposition A. 4. For any e > there exists a finite-rank kernel T, such that, 

\lIwo[R-T]Iwo\L<e. 

Proof. Lemma A. 3 implies that for any e > we can find > 1 such that, 

\lIwoiR-Icr,RIcJIwo\lo<e/2. 

With this value of tq fixed, note that (A2) implies that for any eo > we can construct 
a kernel T{x,dy) = t{x,y)dy of the form given in (8) such that \t{x,y) — r{x,y)\ < Eq for 
(x, y) G X Cro- In particular, the functions {sj} and the densities of the Ui can be taken as 
indicator functions, so that this is simply the approximation of the continuous function r( • , • ) 
by simple functions. Consequently, 

WlWoiR - T]IWo\l, < + lUWoICro [R - T]ICr,IWo\L 

< £/2 + eo sup {vix)W^ix)) ^"^"^(aj 
<e/2 + eor^^^°'(C.J, 

where ^^''^{Cro) denotes the Lebesgue measure of the bounded set C^g. The right-hand-side is 
bounded by e on choosing eo = [^0 1^^""° i^ro)]~^ ■ ^ 



Proof of Proposition 3.1. Since Proposition A. 4 was proved for an arbitrary function Wq 
satisfying (28), we can take Wq equal to a constant, say w >!. First consider the case k = 1. 
There, applying Proposition A. 4 with e/2 instead of e, we obtain a finite-rank kernel of the 
form T = Yjij fijlci ® Letting £q = T], 

¥>n - ^ollL = \A.^R>^ - T\\, = \IR - T\l = hlwolR - T]Iwo\lo ^ ^/2- 
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Now we define £ = Sq + Iq, "X" i^i, with Co = X \ Ui<j<ArCi. We have £^1 = as required, and 
the following bound holds: 

WD. - £\l, < WD. - £o\l + Mv) ( sup 

On choosing Cq sufficiently large, the right hand side is bounded by e, as required. 

For a fixed, general k we consider the scaled process {Z{t) := ^{t/K) : t > 0} and 
note it satisfies exactly the same assumptions as {<I>(t)}. Also, the resolvent kernel of {Z(t)} 
(corresponding to the parameter a = 1) is kR., so that, as before by Proposition A. 4 we 
obtain the required bound. □ 
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